Papers with SciGen datasets
Arithmetic-Based Pretraining Improving Numeracy of Pretrained Language Models (2023.starsem-1)
Copied to clipboard
| Challenge: | Recent work suggests that pretrained language models perform below their capabilities when applied out-of-the-box on tasks that require understanding and working with numbers. |
| Approach: | They propose an extended pretraining approach that addresses both in one extended step . they propose a novel extended pre training objective called Inferable Number Prediction Task to improve numeracy. |
| Outcome: | The proposed approach improves reading comprehension and inference-on-tables tasks without architectural changes or pretraining from scratch. |